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method extends a recently proposed procedure by Ohsmann (l| to 
an order recursive solution. This method docs not involve any di- 
rect matrix inversion and solves the initial estimate by incorporat- 
ing a lattice algorithm. We also derived a more accurate expression 
for the convergence factor which indicates the convergence per- 
formance of the iteration (4). The second method is a simplified 
iterative procedure for situations where the input is known to be 
white or nonwhite with a correlation coefficient of value a < 1/3. 
This method does not require any matrix inversion and is compu- 
tationally efficient. Simulation results indicate satisfactory perfor- 
mance of both methods. Results have also been presented for iden- 
tification in the presence of noise. 

Appendix 

From (16) we have the matrix Q = + ! - D'^R^)- Let us 
consider that the input jr(Jt) has been obtained as a single pole 
process = <xx{k - I) + w(A), where w(k) is a zero mean 
white Gaussian process with unity variance and a is the correlation 
coefficient. In this derivation, we assume that 0 £ or < I without 
any loss of generality. We shall also consider that the resulting 
autocorrelation matrix is symmetric and Toeplitz. Therefore Q 
has zero entries along the principal diagonal. The characteristic 
equation = | Q - X/«+ J = 0, can be written in the following 
recursive form [71: 

A„.(X) = -|2a^ - X(i + a^)lA„,_,(X) 

- a-(l + \y A„.-2(X), m = I, 2, • • • . M 

(A!) 

whereA_|(X)= I, by definition, and Ao( X) = -X. After a few 
manipulations we can rewrite (A I) as 



AjX)=-[a(l+X)]" 



Xsin(m -h l)x ^ a{ \ + X)sinffu 
sinjc sinjt 

(A2) 

Equation (A2) then yields M + 1 distinct eigenvalues given by [7] 



a' — a cos 



(m -I- l)ir 
M + 2 



-2 ■ 



I - 2a cos 



m = 0, I, 2, 



(A3) 



The maximum eigenvalue corresponds to the case m = 0 in (A3) 
and is given by 



X™. = -2 



I - 2of cos I „ ^ ^ ) + of^ 
\M + 2/ 



2a 
I - a 



(A4) 



where we let M be large such that cos {ir/M -t- 2) « I in (A4). 
The convergence condition requires that | X^^^ | < 1 which yields 
the bound a < 1/3. 
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Weighted Averaging of a Set of Noisy Images for 
Maximum Signai-to-Noise Ratio 

MICHAEL UNSER and MURRAY EDEN 



Abstract— This paper considers the problem of estimating a signal 
from a weighted average of ^ registered noisy obse rvations. A set of 
optimaj^ei ^titT nifCoctticients is determined by maximizing a slgnal-to- 
nolse ratio criterion. This solution can be computed by first standard- 
izing each observation with respect to its first and second moments and 
then evaluating the first eigenvector of the corresponding N x N Inner- 
produfl^atrix. The resulting average is shown to be proportional to 
the first basi s^ vector of the Karhunen-Lofeve transform provided that 
the data has been standardized in an appropriate fashion. The low sen- 
sitivity of this approach to the presence of outliers is illustrated by 
using real electron micrographs of ostensibly identical virus particles. 

I. Introduction 

Given a series of noisy observations of an unknown signal, the 
simplest and often most efficient tioise redu ^Uap technique is av- 
eraging [1» p. 434). An interesting application of this technique is 
correlation averaging, which is now used routinely for improving 
si |nal-lo-noise ratio s of electron micrographs of quasi-periodic ar- 
rays,^r s ets of images of ostensibly identical free standing parti- 
cles t2 ]-[^]. In this approach, smgle unit cetls or individiial par- 
ticle images are brought into tra nslational a n d rotational align ment 
[3J-[5], analyzed using multiva'nate statistical methckls to identify 
subsets o f similar particles and eliminate outlier^ 61. [7J, and fi- 
nally averaged. The quality of the restored signal is typically as- 
sessed in terms of estimated signal-to-noise ratios [4], {8]. 

An important step in this whole procedure — although not usually 
emphasized— is the preprocessing where the individual observa- 
tions arc normalized to compensate for va mnp ac ^'i'^^jup" param- 
ctcrs. This is usually achieved by simple linear rescaling of the 
ffifensity distributions. In most applications, the w cifihting coeffi - 
ci ents are chos en to sta ridardize the ran ge (min/max normaliza- 
tion") or the first and second moments (moment normalization) of 
t he intensity distribution of ea ch observation [9|. 

fiTeffect, during this analysis the signal is estimated from a 
wei ghted average of the initial registered noisy observations. Ob- 
viously, there are many ways of selecting the weighting coefficients 
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oiher than the two fore mentioned preprocessing techniques. In this 
paper, we consider these we l uti ng schemes in terms of their per- 
formance for signal estimatiori7In particular, we derive the optimal 
weighting coefficients that maximize a global signal-to-noise ratio 
criterion. We then discuss the properties of this solution which is 
different from a simple standardization and turns out to be closely 
related to the Karhunen-Lo^vc transform. Finally, wc illustrate the 
use of this technique using simulated and real electron micrograph 
data, and compare its performance with simpler normalization pro- 
cedures. 

II. Thr Optimal Solution 

The following method applies to the processing of a setof ^ 
multidimensional registered noisy observations of a given sigmil. 
For notational simplicity, these measurements are represented as 
vectors in an Af-dimensionai feature space. This formulation re- 
quires the choice of a one-to-one mapping between the discrete 
coordinate system in the canonical representation (for example, the 
raster scan of an image) and the components of these feature vec- 
tors. 

A. Statement of the Problem 

Given a set of N noisy measurements. X = {jC;, ( = I. • • • » 
}, represented as M-dimensional vectors, wc arc seeking coeffi- 
cients { W;, / = 1, • • • , } and { t,. / = I, • • • . }. such that 
the linearly rescaled vectors 

Z, = w;[Xi - bit]: / = I. • - - , ^ (j) 

define a set Z = { z,, / = I, • • • , N } that has a maximum signal- 
lo-noise ratio. In (his expression, 1 is an A/-dimcnsional vector of 
all I 's. The signal is estimated from the ensemble average 

i = - • • = - E = - X w-Xx, - bM (2) 

The mean signal value (ordc component) is given by; 



M k = i 



(3) 



The quadratic signal-to-noise ratio (SNR) is defined by the series 
of equations 



SNR 4 - 



(4) 



(5) 



^ ^ 



(6) 

In other words, and £.„ are the rescaled signal and residual 
noise energies, respectively. The factor A^ in (4) accounts for the 
fact that the noise is expected to be reduced by A^ in the final aver- 
age. The operator denotes the transpose of a matrix and || . ||- 
is the square norm of a vector. Note that the dc signal component 
is not taken into account in the definition of the signal energy. 

B. Optimal Additive Coefficients 

Fortunately, the determination of the optimal additive and mul- 
tiplicative coefficients in { I ) can be carried out independently. This 
is a consequence of the following theorem. 

Theorem 1: For any given set of weights { h-,, / = I . • • • . A^ }, 
the A/s in (1) that maximize the SNR defined by (4)-(6), are given 
by 



^i = ~'i = t: 2 ^.t; / - I, • • • . A^ 
M k-i 

and correspond to the average signal value of each vector. 



(7) 



Proof: Lets us define the centered variables 

«, = "vlx,- - 1,11; I = I. • • • , Af (8) 

which arc associated to the average signal u. It is simple lo show 
that 

s = lr'5 = o 

and to express the estimated signal z as 

u = S + EL (9) 

By sub.stituting this expression in (5). the signal energy may be 
rewritten as 



l|i-3l|r = 



(10) 



and is independent of the value^of the 6;\s. Similarly, by using (9) 
and the fact that Zi = «, + *v;Ijc; - 6; 1 1, the noise energy is equal 
to 



is 



Since the vectors Wi. 



. U\; and u arc all orthogonal to 1. i.e.. 



u, 1 = Mui — 0, this expression is further decomposed as 



(II) 

Cleariy. E„„ is independent of the 6/'s. Therefore, for any given 
weights {wj, i = I, • • • , W }, the SNR is minimum when £„ is 
minimum. It is now straightforward to show that the lowest pos- 
sible value of Eq, namely Eq - 0. is reached when the bi's are 
selected according to (7), which also results in z = 0. Obviously, 
wc can also add any fixed constant to the expression given by (7). 

C. Optimal Multiplicative Coefficients 

We now proceed to determine the optimum w,*s by replacing the 
fr,'s in ( I ) by their optimal values: We define {he N x N centered 
inner product matrix 

^,vi = [/-y] with nj ^ xjxj - MxiXj (12) 
and the corresponding diagonal matrix 



U = 0. 



(13) 



Z)„ = \d,j] with , ^ ^ ^ 

(' ^j) 

The main result of the paper is the following. 

Theorem 2: For additive weights in (I) given by (7), the vector 
of multiplicative coefficients w — [u-, * • • u'vj'^thai maximizes 
the SNR defined by (4)-(6) is the first generalized eigenvector of 
the characteristic equation: 

R,^,w = ^D,,w, (14) 
In other words, w is an element of the null space of ( R, , - j3/), , ). 
with /3 the largest value such thai there is a nonzero nul space. 

Proof: By using (10), we show that the rescaled signal energy 
is given by 



(15) 



Similariy. using the centered variables defined by (8). the total 
energy is equal to 



I 



(16) 



and can be decomposed as 
I ^ n I 



i«,ir 



\\uW = + 



(17) 
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Consequently, maximizing (E^/E„) is equivalent to maximizing 
{E.jE.j). In addition, the SNR is independent of any global scal- 
ing factor so that we can fix the total signal energy to some arbitrary 
value: 

Ea = Jj ^^^xx"^ = const (18) 

and simply maximize E.^ - w^i?,, w under this constraint. The so- 
lution to this optimization problem is found by defining the auxil- 
iary function 

L{yf. 0) = w^R,,w - &lw^D,,w - const] (19) 

where ^ is a Lagrange multiplier, and by setting its partial deriv- 
atives with respect to w to zero, which yields (14). 

The first eigenvector of (14) is the one that corresponds to 
the largest eigenvalue: 

0^ = max {{w'^R,,w)/iw^D,,w)} . (20) 

By decomposing the denominator of this expression into its signal 
and noise energy components, we find that (he maximum signal- 
lo-noise ratio is given by 

max{SNR}-— (21) 
/V — Pi 

D. Relationship With the Karhunen-Loeve Transform 

The process of finding the optimal weights is simplified when 
is proportional to the identity matrix. This is precisely the case 
for the normalized variables: 

where a is an arbitraiy multiplicative factor, and j, the mean vector 
value defined by (7). In this particular case, we show that (14) is 
equivalent to 

RyyV = \v (23) 

where v denotes the vector of weights to be applied to the^/'s. The 
optimal weights are given by the first eigenvector V\ of the nor- 
malized inner product matrix Ryy - Y^Y, where V = (yi * ' • ys] 
is the M X N normalized data matrix. In addition, we can show 
that the optimally weighted average z can be directly determined 
from the Karhunen-Lofeve (or principal component) representation 
of the normalized data. More specifically, we would like to em- 
phasize the following properties. 

Property /: The projection of the data matrix on the first eigen- 
vector of /f, v is proportional to z. 
Proof: 

s 

Yvi - 2 V\,yi = A/z. 

Property 2: z is proportional to the first eigenvector (or eigen- 
signal) of the Af x M scatter matrix Syy = YY^. 
Proof: Using (23), we have that 

Y^Yv^ = X,i;, 

which we multiply on the left by Y 

Y{Y''Y)vi = X,yi;, 
{YY^){Yv,) = X,(y£;,). 
By using Property 1, this last expression is also equivalent to 
{YY')z = Kt 

Property 3: The nonzero eigenvalues { X;, / = 1 , • • • , /V } of 
Ryy = y'^y (or 5,,. = yy'^) are directly proportional to the eigen- 



values of (14) and the optimum SNR is given by 

max {SNR} (24) 

(-2 

Proof: Equation (22) may be rewritten as ys - aZ),",'''^|x, - 
x, I ], which directly implies that 

Ryy = a'-D::^'R„D:y\ 

Substitution of this expression in (23) and left multiplication by 
ia--D\(') yields 

R..D;y'v = {\/a')D\Pv = {\/a-)D.AO:r-v]. 
If we now define w = D '^'v, we find that 
R„w - i\/a')D,,w 

which is equivalent to (14) provided that P = X/a". We then sub- 
stitute this value in (21) and note that tr {Ryy) = a^N = E-Li X„ 
which finally yields (24). 

E. Comments 

The maximum signal-to-noise ratio scaling method (MSNRS) 
was derived by maximizing a global SNR criterion that is entirely 
specified by the data to be analyzed. This performance criterion, 
which is only an approximation of the true signal-to-noise ratio, 
has been used previously to assess the efficiency of correlation 
averaging techniques [4], {8]. An important point is that the present 
method makes no assumptions about the underiying signal model 
and, in this sense, is nonparametric. 

From the point of view of statistical estimation theory, a more 
satisfactory signal estimate would be the one with minimum mean 
square error (MMSE), which is briefly reviewed in Appendix A. 
The major difference between MSNRS and MMSE averaging is 
that the latter is parametric and requires the knowledge of the noise 
statistics and the scaling coefficients for each image. The MMSE 
estimation technique is therefore not directly applicable in practice, 
unless the parameters a and C^j which appear in (A6) are estimated 
based on some initial approximation of the signal. Despite this dif- 
ference in philosophy, the equations defining either of these solu- 
tions ((14) and (A9), respectively) are structurally very similar, as 
emphasized in Appendix A. 

III. Experimental Results 
A set of negatively stained capsomer images of herpes simplex 
virus (HSV) [10] has been used to illustrate the application of our 
algorithm. This data was obtained using high resolution electron 
microscopy. The original micrographs were digitized with a Per- 
kin-Elmer scanning microdensitometer with an effective sampling 
step of 0.3 nm. Individual capsomer subimages were extracted and 
brought into rotational and translational alignment using correla- 
tion techniques 12]-151. Fig. 1 displays the subset of 30 such reg- 
istered subimages that were used for our analysis. After compari- 
son of the various weighting schemes for the analysis of this 
particular data set, the extracted signal was used as a test object to 
further investigate the performance of these algorithms using var- 
ious levels of Gaussian noise. 

A. Analysis of Experimental Electron Micrographs 

The original data set was linearly rcscaled using (1) with weight- 
ing coefficients determined according to the three following pro- 
cedures: 

1) Minimum/maximum normalization (MMN): The minimum 
and maximum gray level values are constant across alt images. 

2) Moment normalization (MN): The mean gray level value and 
energy are constant across all images which implies that the various 
feature vectors have a constant norm. 

3) Maximum SNR scaling (MSNRS). 

The resulting averages (N = 30) arc shown in Fig. 2 with their 
corresponding signal-to-noise ratios estimated using (4)-(6). Al- 
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Fig. 1. Sci of 30 expcrimcnial 50 x 50 op.sonicr invd^cs ol' herpes 
simplex vim.s type II. 




40 5^:+: :,7?t; :>2% 



Fig. 2. Compurison ol'diflereni .signal esiimalion icchniqucs. (a) Average 
obtained using range mirmalizatinn. tb) average obtained using momcm 
nnrmalization. ic) maximum signul-to-miise ratio average. Td) 5 first 
eigcn-images of the KLT (or standardized imiiges with their correspond- 
ing relative energy contributions (eigenvalues!. 



though these averages are very similar visually, there arc differ- 
ences in the values of the signal-io-noisc ratio. The worst results 
are obtained using the min/max normalization. This is not very 
.suiprising since the minimum and maximum values of an image 
are known to be quite sensitive to the pre.sence of noise. The mo- 
ment normalization improves the SNR by 11.5%. The MSNRS 
provides a funher 4% increase which is consistent with our theory. 
We also computed the Karhuncn-Lo5vc transform in the case of 
the images standardized according to the second procedure. The 
corresponding cigenimagcs arc shown in Fig. 2(d). The first eigen- 
image is identical to that of Fig. 2(c) and is therefore also very 
similar to the other averages. We can also verify that the value of 
the maximum SNR predicted by (24) is identical to the value as- 
sociated with the optimal average in Fig. 2(c). 

B. Outlier Ideiuifivalion 

The main advantage of the MSNRS is that it decreases the sen- 
sitivity of the analysis to the presence ofoutlicni. To illu.stratc this 
effect, we analyzed our data using the odd men out (OMO) algo- 



rithm |7|. This procedure uses a criterion of mutual statistical con- 
sistency in order to rank the data from most to lea.st reliable. Out- 
liers arc then identified by imposing an acceptability threshold on 
this ordered list. Fig. 3 represents the optimal multiplicative 
weights as determined by (22) and (23) as a function of the rank 
produced by the OMO algorithm. The weights consistently de- 
crease as a function of reliability. The Spearman rank correlation 
is as good as p = -0.977 indicating that both methods are in ex- 
cellent agreement. The last six images, which arc identified as 
numbers 14. 24. 28. 27. 23. 25 in Fig. I. are distinguishable Irxmi 
the bulk of the data in that they are given substantially lower 
weights. These images probably correspond to outliers, which is 
fully consistent with the outcome of the OMO algorithm. 

The conventional approach to signal extraction using correla- 
tion-averaging techniques is to identify outlier; using multivariate 
statistical analyses and to exclude those images from the final aver- 
age 16], 171. [ 1 1 1. For this particular example, the OMO algorithm 
rejects precisely the 6 images identified above I7|. The signal es- 
timate obtained by averaging the remaining 24 images has an SNR 
of 19.57. This value is extremely clo.sc to the one obtained by 
averaging the full set of 30 images using MSNRS. although the 
two approaches are ba.scd on very dilfercnt principles. 

C. Simulation Experiments 

To compare the perfomiance of these algorithms in terms of a 
fully defined model system, wc used the average HSV capsoincr 
image in Fig. 2(c) and contaminated it with increasing levels of 
noise. We performed two independent experiments using white 
Gaussian and uniformly di.stributed noi.se. For this purpose, wc 
generated a sequence of 20 test iniages with a quadratic signal-to- 
notse ratio or - \/k-. where k is the rank of the corresponding 
image. The first 5 of these images in the first experiment are dis- 
played in Fig. 4. The averages obtained by using no scaling at all. 
MMN. MN. and MSNRS. respectively, arc shown in Fig. 5. The 
corresponding mca.sures of the SNR obtained using f3)-(6) are 
given in Table I. For comparison, we also computed the true 
signal-to-noise ratio (or) of these averages using the noncorrupted 
signal (Fig. 2(b)) as reference. Clearly, the best results arc ob- 
tained for the MSNRS which tends to give the highest weights to 
the le.ss noisy measurements. The corresponding value of the true 
SNR (a = 1.257 and 1.29) are not too different from the theoret- 
ical optimum (ininimum mean square error estimate) that could, in 
principle, be obtained by weighting the images by the inverse of 
their true variance {a = {.5962, as computed from (A8) in Ap- 
pendix A). Otherwise, for all other methods, the quality of the sig- 
nal estimate is substantially below that of even the least noi.sy im- 
age of the set. All scaling algorithms performs cs.seniially the same 
for Gaussian and uniform noise, with the exception of MMN which 
is slightly less performam for the latter case. 

For this particular example, it is interesting to note that the SNR 
is a rather optimistic estimate of the true signal-io-noise ratio, par- 
ticularly for those methods that perform very poorly. This follows 
from the fact that a badly estimated signal tends to tie much clo.ser 
on average to the majority of noisy measurements than does the 
uncorrupted signal. As a consequence, the residual noise variance 
is usually underestimated leading to an SNR value that can be sub- 
stantially above the true value. Not surprisingly, the more realistic 
SNR estimate is obtained for the better performing method, namely 
the MSNRS. 

D. Discussion 

In the first example, the averages obtained using either scaling 
method arc very similar perceptually. However, our quantitative 
analysis shows that the result obtained using the MMN is the less 
favorable one. The performance of the MN is generally quite 
satisfactory and the decrement in performance with respect to our 
optimal approach is only a few percent. We have also performed 
this evaluation using different sets of micrographs such as the one 
studied in |4] and 1 12], and have obtained quite comparable results. 
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Fig. 3. Maximum signal -to- noise ratio weights for standardized experi- 
mental capsomer images in relation to the rank obtained with the OMO 
algorithm. 




Fig. 4. First 5 images of a total .set of 20 HSV capsomcrs overlaid with 
increasing amounts of computer-generated Gaussian noise. Signal-lo- 
noise ratios: at = \/k', k = 1. * • • .20. 



Fig. 5. Comparison of different signal estimation techniques for a gener- 
ated data set with increasing levels of Gaussian noise, (a) Initial image 
average, (b) average obtained using range normalization, (c) average ob- 
tained using moment normalization, (d) maximum signal-to-noi.se ratio 
average. 



example is that the relative increase of the true signal-to-noisc ratio 
may be substantially greater than the corresponding improvement 
of the SNR criterion (cf. Table I). As pointed out eariier, this is 
due to the fact that the noise component lends to be underestimated 
for poor signal estimates. This result suggests that for real micro- 
graphs such as those shown in Fig. L the improvement of MSNRS 
over the other schemes may be even greater than that which may 
be inferred from the relative values of the SNR criterion. This as- 
senion. however, remains speculative since there is no way to mea- 
sure the true signal-to-noise ratio for real noisy data sets. 

Based on these observations, it appears that the simple MN is 
adequate for analyzing reasonably homogeneous data sets; that is. 
apart from some constant scaling factors, the individual observa- 
tions are approximately identically distributed. The MSNRS is the 
most useful in less ideal cases when the data at hand is likely to 
contain outliers or when the individual observations are subject to 
different levels of perturbation. 

In contrast with the OMO algorithm 17] which operates on a 
simple acceptance/ rejection basis, the MSNRS treats outliers in a 
graded manner and is closer to the principle underiying fuzzy set 
theory. In this sense, the weight given to a particular image ex- 
presses our confidence that it belongs to the main group. We also 
remark that unlike the OMO algorithm our present derivation makes 
no assumptions about the underiying signal model. 

According to property 2, when the data has been properiy nor- 
malized the first eigenvector of the KLT is proportional (o the max- 
imum signal-to-noise ratio signal estimate. This result provides a 
mathematical explanation for the great similarity that can usually 
be observed between the first basis vector of the Karhunen-Lofeve 
transform (or principal components) and the average of the data. 
In fact, the difference between those two vectors is significant only 
when the MSNRS clearly outperforms the MN. Conversely, for 
very neariy homogeneous data sets (e.g. images with a common 
mean and identically distributed noise), the first eigen-coordinates 
should be essentially a constant across all images. 

Finally, we remark that in many applications of KLT or princi- 
pal components analysis the main group effect is intentionally fac- 
tored out by subtracting the mean vector and performing the anal- 
ysis on centered variables. Our study provides mathematical 
evidence to justify the physical judgments that underiy the use of 
this factor. 



TABLE I 

Pkrforman'CE Assessment of Various Scaling Methoi>s for the 
Average OF 20 Test Images With Increasi.ng Levels of 
Gaussian and Uniform Noise 



Scaling method 


SNR criterion 
(Gauii, Unifonn) 


True ti(Ral-to-rKMse mio (a) 
(G»uii, Uniform) 


none 


(L16. 1.21) 


(0.14. 0.15) 


nonnalization (MMN) 


. (1.53. 1.48) 


(0.53. 0.45) 


moment nomuliialkm (MN) 


(1.57. 1.37) 


(0.53. 0.56) 


maximum SNR tcaling 
(MSNRS) 


(1.96. 1.94) 


(1.26, 1.29) 



In analyzing these data sets, we observed that the main advantage 
of the MSNRS lies in its ability to handle outliers by decreasing 
their relative contribution to the final average. 

To further differentiate between these approaches, we have con- 
sidered the simulated example of a signal corrupted by various lev- 
els of Gaussian noise. For this extreme case, the MSNRS alone 
resulted in satisfactory signal extraction, owing to the fact that it 
weights the images approximately in proportion to their signal-to- 
noise ratio. The performance of the other weighting schemes was 
less than adequate, with a signal estimate that was even noisier than 
the best image of the set. An important point brought out by this 



IV. Summary and Conclusion 

In this paper, we have considered the problem of estimating a 
signal from a weighted average of N noisy observations or feature 
vectors. In this approach, the intensity distribution of each obser- 
vation is scaled linearly to compensate for varying acquisition pa- 
rameters. The optimal weights have been derived by maximizing a 
global signal-to-noise ratio criterion. The determination of the 
MSNRS solution involves the standardization of the individual ob- 
servations by subtraction of the mean intensity value and division 
by the square root of the residual energy. The optimal weights are 
then obtained from the components of the first eigenvector of the 
corresponding inner-product matrix. 

An interesting conceptual result is the link that has been estab- 
lished between the first basis vector of the Kartiunen-Lofeve trans- 
form for normalized feature vectors and the average vector: both 
of these vectors are estimates of the underiying signal with the for- 
mer being the one with maximal signal-to-noise ratio. 

The MSNRS provides an upper bound on performance for com- 
parison with other simple scaling methods. We have shown that its 
most interesting feature lies in its capacity to adjust to the presence 
of outliers. It is also particularly suited for analyzing data sets of 
uneven quality where the initial signal-to-noise ratio can vary sub- 
stantially from one observation to another. Other than that, for rea- 
sonably homogeneous data sets, a simple normalization (mean sub- 
traction and constant norm) of the feature vectors prior to averaging 
leads to a signal estimate that is only slightly suboptimal. The 
MSNRS is therefore best suited for what we have referred to as less 
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than ideal cases. A good example of application is high resolution 
electron microscopy where it is difTicult to collect images that are 
fundamentally alike due to differences in the levels of noise, vari- 
ation of viewing geometry, nonuniformiiy of staining, or other 
structural penurbaiions. 

Appendix A 
Minimum Mean Square Error Averaging 

In this Appendix, we derive the multiplicative weights for min* 
imum mean square error averaging, assuming that the noise statis- 
tics as well as scaling factors associated with each observation are 
known. For simplicity, wc consider the case of one-dimensional 
feature vectors. The results are easily extended to higher dimen- 
sions by replacing the /i" by the norm of the signal vector and the 
bctween-measuremeni covariance matrix C, , by the average of these 
matrices across all vector components. 

Given a set of N noisy measurements {x, } such that 

EM=^a;ti (Al) 

where { a, } is a set of known constants, the problem is to determine 
an estimation of the signal parameter n from the weighted sum: 

A = £ H-.Jr, (A2) 

with the requirement that 

f{A}=M (A3) 

such that the mean square error t- = £"{ ( ^ - } is minimized. 
Wc note that this solution will also maximize the signal-io-noise 
ratio defined as a = n'/f'. The first condition implies that 

s 

2 WfOi = w^a = I (A4) 
/" t 

where w = Iw, • ■ w^v^and o = |a, • • - t/^vl ^ It is straight- 
forward to show that the residual error is given by 

e- = wX..w (A5) 

where C,, = [c^j] is the betwecn-measuremcnt covariance matrix 
whose entries arc defined as: 

c, = E{(.,-E{x,]){xj-E{xj})}. 

Taking the partial derivatives of (A5) and (A4) with respect to h*, 
we find an expression that is consistent with standard results on 
linear unbiased minimum variance estimation (13, ch. 5|: 

w = XC-'fl (A6) 

where X is a Lagrange multiplier which should be chosen to satisfy 
(A4). If the measurements are mutually uncorrelaied, we get the 
well-known result that the observations should be given a weight 
that is inversely proportional to their variance: 

IV, = (a;X)/Q. (A7) 

It is relatively easy to show thai the corresponding maximum SNR 
is given by: 

a* - L oti (A8) 

where a, = {aiPL)/ci, is the SNR associated to the random variable 
Xi. We now rewrite (A6) to bring out the similarities with (14). The 



simplest form of (A6) is 

C,,w = Xfl 

which, by noticing that aa^w is always in the direction of a, is aLso 
equivalent to 

ii\aa^)w = 0[C„ + t^-{aa^)\w, (A9) 

Equation (A9) is now in a form that is comparable with (14). In 
particular, the matrix which is of rank I. is the para- 

metric equivalent of the matrix in (14). For instance, the ex- 
pected signal energy is given by: 

E{ii}" = w^\ii-{aa^)]w = {p^w'af (AlO) 

which is similarto (15). Wc also note that (C,, -H /i-(aa' )| is the 
parametric equivalent of Z>, ^ since the total energy of the estimated 
signal is given by 

£{A'} = + £{ A}' = y^^[C.. + A<^^)\^ (All) 
which is similar to ( 1 9). 
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